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Abstract 

Classical results for exchangeable systems of random variables are extended to 
multi-class systems satisfying a natural partial exchangeability assumption. It 
is proved that the conditional law of a finite multi-class system, given the value 
of the vector of the empirical measures of its classes, corresponds to independent 
uniform orderings of the samples within each class, and that a family of such 
systems converges in law if and only if the corresponding empirical measure 
vectors converge in law. As a corollary, convergence within each class to an 
infinite i.i.d. system implies asymptotic independence between different classes. 
A result implying the Hewitt-Savage 0-1 Law is also extended. 
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1. Introduction 

Among many others, Kallenberg [3], Kingman |10| . Diaconis and Freedman 0], and 
Aldous pQ study exchangeable random variables (r.v.) with Polish state space. The 
related notion of chaoticity (convergence in law to i.i.d. random variables) appears in 
many contexts, such as statistical estimation, or the asymptotic study of interacting 
particle systems or communication networks. It is behind many fruitful heuristics, such 
as the "molecular chaos assumption" (Stosszahlansatz) used by Ludwig Boltzmann to 
derive the Boltzmann equation, see Cercignani et al. [3J Sect. 2, 4]. 

A sequence of finite exchangeable systems converges in law to an infinite system if 
and only if the corresponding sequence of empirical measures converges to the directing 
measure of the limit infinite system, given by the de Finetti Theorem. Hence, chaoticity 
is equivalent to the fact that the empirical measures satisfy a weak law of large numbers, 
for which A.S. Sznitman developed a compactness-uniqueness method of proof yielding 
propagation of chaos results for varied models of interest. Sznitman also devised a 
coupling method for proving chaoticity directly. See Sznitman [13j for a survey, and 
Meleard [TT] and Graham [51 [5J for some developments. 

The above notions pertain to the study of similar random objects, but many sys- 
tems in stratified sampling, statistical mechanics, chemistry, communication networks, 
biology, etc., involve varied classes of similar objects (which we call "particles"). See 
for instance Cercignani et al. [3J ("Mixtures", Subject index p. 454) and the review 
papers [BJ [SJ [T2] in a recent book. 



* Postal address: CMAP, Ecole Polytechnique, CNRS, 91128 Palaiseau France. 



1 



2 



Carl Graham 



Our paper considers natural notions of multi-exchangeability and chaoticity for 
such multi-class systems, and extends the above results. These notions are explicit 
in Graham [SJ pp. 78, 81], and implicit in [31 ^ [5J [T2] where the corresponding limit 
equations are directly considered. Graham and Robert [7] extend Sznitman's coupling 
method in this context. For infinite classes, Aldous calls multi-exchangeability "internal 
exchangeability" just before [TJ Corollary 3.9]. 

We prove that the conditional law of a finite multi-class system, given the value of 
the vector of the empirical measures of its classes, corresponds to choosing independent 
uniform orderings of the samples within each class, and that a family of such systems 
converges in law if and only if the corresponding empirical measure vectors converge 
in law. We conclude by extending a result implying the Hewitt-Savage 0-1 Law. 

As a corollary, for a multi-exchangeable system, chaoticity within classes implies 
asymptotic independence between classes, see Theorem [3] below. This striking result 
allows rigorous derivation of limit macroscopic models from microscopic dynamics using 
Sznitman's compactness-uniqueness methods, and was a major goal of this paper. 

We state as a "Proposition" any known result, and a "Theorem" any result we 
believe to be new. All state spaces S are Polish, and the weak topology is used for the 
space of probability measures V(S) which is then also Polish, as are products of Polish 
spaces. For k > 1 we denote by the set of permutations of {1, ... ,k}. 



2.1. Finite and infinite exchangeable systems 

For N > 1, a finite system (X^)i< n <N of random variables (r.v.) with state space 
S is exchangeable if 



Then, the conditional law of such a system given the value of its empirical measure 



corresponds to a uniform ordering of the ./V (possibly repeated) values occurring in A N 
(its atoms, counted according to their multiplicity), see Aldous [1, Lemma 5.4 p. 38]. 

An infinite system (A„)„>i is exchangeable if every finite subsystem (X n )i< n <N is 
exchangeable. The de Finetti Theorem, see e.g. [9l QUI HI [I] , states that such a system 
is a mixture of i.i.d. sequences: its law is of the form 



where C\ is the law of the (random) directing measure A which can be obtained as 



2. Some classical results 




(1) 





(2) 



Thus, laws of infinite exchangeable systems with state space S and laws of random 
measures with state space V{S) are in one-to-one correspondence. 
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All this leads to the following fact, see Kallenberg Theorem 1.2 p. 24] and 
Aldous [1, Prop. 7.20 (b) p. 55]. 

Proposition 1. Let (X^)i< n <N for N > 1 be finite exchangeable systems, and A N 
their empirical measures ([I]). Then 

lim (X%)i<n<N = (X n ) n >i in law, 

N—>oo 

where the (infinite exchangeable) limit has directing measure A, if and only if 

lim A N = A in law. 

TV— >oo 

A sequence (Xn)i< n <N for N > 1 is P-chaotic, where P € V(S), if 
lim C(X?, Xf ) = P® k , V* > 1 , 

N^oc 

i.e., if it converges in law to an i.i.d. system of r.v. of law P. The following corollary 
of Proposition [1] is proved directly in [T31 Prop. 2.2 p. 177] and [TTJ Prop. 4.2 p. 66]. 

Proposition 2. Let (X^ )i< n <jv /or ./V > 1 6e finite exchangeable systems, A N their 
empirical measures and P G V{S). Then, the sequence is P-chaotic if and only if 

lim A N = P in law 

N—hx 

and hence in probability, since the limit is deterministic. 

2.2. Multi-exchangeable systems 

We assume that C > 1 and state spaces Si for 1 < i < C are fixed. For a multi-index 
N = (Ni)i<i<c £ N c we consider a multi-class system 

(X™i)i< n <Ni, i<i<c , X™i with state space <S;, (3) 

where X^ i is the n-th particle, or object, of class i, and say that it is multi-exchangeable 
if its law is invariant under permutation of the particles within classes: 

£{( X ™(n),i)l<n<Ni,l<i<c) = £{(X™i)l<7i<Ni,l<i<c) , VCTj £ S(iVj) . 

This natural assumption means that particles of a class are statistically indistinguish- 
able, and obviously implies that {X^ i ) 1 < n < : N i is exchangeable for 1 < i < C. It is 
sufficient to check that it is true when all a but one are the identity. We introduce 
the empirical measure vector, with samples in V(S\) x • ■ ■ x V(Sc), 

, Ni 

(A?)i<i<c, A* = — (4) 

2 n— 1 

We say that the multi-class system (X n i) n >i,i<i<c with infinite classes is multi- 
exchangeable if every finite sub-system (X nt i)i< n <Ni, i<i<c is multi-exchangeable. Par- 
ticles of class i form an exchangeable system, which has a directing measure Aj, and 
we call (Ai)i<i<c the directing measure vector. 

The following result is given in Aldous |TJ Cor. 3.9 p. 25] and attributed to de Finetti. 
A remarkable fact is conditional independence between different classes. 
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Proposition 3. Let (-X' n ,i) Tl >i ) i<i<c be an infinite multi- exchangeable system, and hi 
be the directing measure of (X n ^) n >i. Given the directing measure vector (Aj)i<j<c, 
the X n> i for n > 1 and 1 < i < C are conditionally independent, and X n j has 
conditional law Aj. 



We shall extend to multi-exchangeable systems the main results for exchangeable 
systems, which hold even though the symmetry assumption and resulting structure 
is much weaker. Indeed, the symmetry order of the multi-exchangeable system is 
Ni ! • • • Nc ! whereas the symmetry order of an exchangeable system of same size is the 
much larger (Ni H \- N c )l 

The following extension of [I] Lemma 5.4 p. 38] (stated in words at the beginning 
of Section [5]) shows that, for a finite multi-exchangeable system, the classes are con- 
ditionally independent given the vector of the empirical measures within each class. 
Hence, no further information can be attained on its law by cleverly trying to involve 
what happens for different classes. 

A statistical interpretation of this remarkable fact is that the empirical measure 
vector is a sufficient statistic for the law of the system, the family of all such laws 
being trivially parameterized by the laws themselves. 

Theorem 1. Let (Xni)i< n <Ni, i<i<c be a finite multi- exchangeable system as in ([3]). 
Then its conditional law, given the value of the empirical measure vector (A^)i<i<c 
defined in corresponds to independent uniform orderings for 1 < i < C of the Ni 
values of the particles of class i (possibly repeated), which are the atoms of the value 
of Af (counted with their multiplicities). 

Proof. Multi-exchangeability and the obvious fact that 



imply that for all g : x • • • x V{S C ) -> R+ and /, : Sp -> R+ we have 



3. The extended results 





c 



E g((Af) 1 < J < c )Y[MX",...X™J 



c 



Nt\ ^ "' N c \ 



£ e ff((Af) 1£ ,< )n/i(^ (1) ,„-^ 




o-iGB(iVi) o- c GS(JV c ) 



C 



1 



E g((Af )k 3 -< C ) J] 



i=l 



CTeS(JV<) 




1 
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where the empirical measure 



NA E ^ x ^<i),i'---' x ^(iv 1 ) 



corresponds to exhaustive uniform draws without replacement among the atoms Xf^, 
. . . , Xj}. i of Af counted according to multiplicity, and hence is a function of Af. 
Since g is arbitrary, the characteristic property of conditional expectation yields that 



E 



c 



Ki<C 



o-es(JVi; 

which finishes the proof, since the /$ are arbitrary and the spaces Polish. 



This result and Proposition [3] lead to the following extension of Proposition [TJ We 
denote by liniN->oo the limit along a fixed arbitrary subsequence of N € N c such that 
mini<i<c Ni goes to infinity. 

Theorem 2. We consider a family of finite multi-exchangeable multi-class systems 

{Xn,i)l<n<Ni, l<i<C j NeN , 

aZZ o/ t/ie form ([3]) luii/i i/ie same C > 1 and siaie spaces Si, and the corresponding 
empirical measure vectors (Af r )i<i<c given in l[4|). Tften 

lim (A^Ji^n^Af;. i<i<c = (X n i) n >i, i<(<c inlaw, 

N^oc ' 

where the (infinite multi-exchangeable) limit has directing measure vector (Aj)i<j<c, 
j/ and on/?/ «/ 



lim (Aj )i<i<c = (Ai)i<,<c in Zaw. 



N 



Proof. Since the state spaces are Polish, it is enough to prove that for arbitrary 
k > 1 and bounded continuous /j : <S- — ► M for 1 < i < C we have 



lim E 

N^oc 



E 



if and only if 



lim E 

N^oc 



E 



J { fi(Xl,i, ■ ■ ■ Xk,i) 



(6) 



(7) 



Let (m)fe = ^fzWj = w(m — 1) • ■ ■ (m — fc + 1) for m > 1 and, for Ni > fc, 

1 



AP = 



(JVi), 



E 



l<ni , . . .,rifc <7Vi 
distinct 
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denote the empirical measure for distinct fc-tuples in class i, corresponding to sampling 
k times without replacement among Xf % , ... , X^ z t . Theorem □ implies that 



E 



c 



E 



E 



Hi V,^' V 

»=i 



(A 



N i 



)\<i<C 



(which follows directly from {J} with g — 1 and the extensions of f. t on S, 
Proposition [3] similarly implies that 



E 



■ Xk,i) 



i=l 



n<^ A ? fc > 



(8) 



and 



(9) 



The corresponding empirical measure for sampling with replacement is given by 

1 



(Af) 



E 



1 l<m,...,n k <Ni 

1 4 l<m,...,n fc <iVi 

not distinct 



and in total variation norm ||/x|| = sup{ (<ft, /i) : ||</>||oo < 1 } we have 



(Af) 



A 



N,fc 



<2 Nj-(Nj)k < fe(fc-l) 



(10) 



where we bound — (Ni)k by counting k(k — l)/2 possible positions for two identical 
indices with JVj choices and N^~ 2 choices for the other k — 2 positions. Hence, if ([6]) 
holds then ©, (HUJ) and © imply ©, and conversely, if © holds then (HH]), © and 
(O imply ([6]), which concludes the proof. 

Let Pi G V{Si) for 1 < i < C. We say that the family of finite multi-class systems 
such as in Theorem [2] is (Pi, . . . , Pc)-chaotic if 



lim £((X n N i )i<„< M <K C )=Pf 



P, 



C ' 



Vfc > 1 



This means that the multi-class systems converge to an independent system, in which 
particles of class i have law Pi. We state a striking corollary of Theorem [2j 

Theorem 3. We consider a family of finite multi- exchangeable multi-class systems 
such as in Theorem^ and Pi e V{Si) for 1 < i < C. Then the family is (Pi, . . . , Pc)- 
chaotic if and only if the (X™ i )i< n <:N i are Pi-chaotic for 1 < i < C . 

Proof. Since (P)i<i<c is deterministic, lim N ^ 00 (Af I )i<i<c = (P)i<i<c in law if 



and only if lim]\j_ 



A 



N 



Pi in law for 1 < i < C. We conclude using Theorem [2] 
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We finish with the following extension of Aldous [H Cor. 3.10 p. 26] and of the 
Hewitt-Savage 0-1 Law. For k > 1, we say that a set 

B c S%° x • • ■ x Sg? 

is fc-multi-exchangeable if for all permutations <7j of {1, 2, . . .} leaving {k+ 1, k + 2, . . .} 
invariant, 1 < i < C, we have 

(^n,i)n>l, l<i<C G fl (^o-i(7i),i)n>l, l<i<C € . 

We define the multi-exchangeable c-algebra 

£ = P| , £ fe = {{(A, M )„>i, i<j<c E 5} : 5 is fc-multi-exchangeable} , 

fe>i 

and multi-tail cx-algebra 

^ = fl ^ k ' ^fc = cr ((^n,i)n>fc, l<i<c) ■ 

fc>l 

Clearly, 7fe + i C £k and hence T C £. 

Theorem 4. Lei (-X" ni j) n >i i<j<c be an infinite multi- exchangeable system with di- 
recting measure vector [^-i)\<i<c- Then 

cr((Aj)i<i<c) = T= £ a.s. 

If moreover the X n ^ are independent, then P(A) E {0, 1} for all A £ £ . 

Proof. Consideration of (f5|) yields cr((Aj)i<j<cr) C T, a.s., and we have seen that 
T C £, hence the first statement is true if £ C er((Aj)i<,-<c), a - s - Now, let A € £. For 
every fc > 1, since A E there is some fc-multi-exchangeable set such that 

^4 = {{X n ,j)n>l, l<j<C & Bk} 

and hence, for all permutations o~i of {1, 2, . . .} leaving {fc + 1, fe + 2, . . .} invariant, 

(1,4, -^(^(rO.OnM, l<i<C — (ls fc ((A„ J )„>i i l<j<c), A CTi („) ,i)rj>l, l<i<C 

— (ls fc ((A' (Tj .(„)j)„>i i K)'<c))-^ff,(n),i)n>l, l<i<C 

and the multi-exchangeability of (A„.i)„>i. i<i<c implies that 

£((l J 4,^ - j (n),i)n>l ; l<i<c) = £(0-B k ((^ (n),i)n>l, l<J<c), ^a 4 (n),i) n > 1; i<j< C ) 

= ^((ififc ((A„j)n>l, l<j<c), X n,i) n > lt i<i< C ) 
= £((lA,^n,j)n>l, KKCj ■ 

Thus (1a, A„.i)„>i : i<i<c is infinite multi-exchangeable, and Proposition[3]implics that 
the (1a, X nt i) are conditionally independent given (Aj)i<j<c and have conditional laws 
Aj, where considering |(5J) we have 

1 w 1 W 

n— 1 n— 1 
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Hence, for arbitrary k > 1 and Borel sets B n j C Si for 1 < n < k and 1 < i < C, 

P (X„ ti e B nti :l<n<k,l<i<C\A, (A,)i<*<c) 

= P (X n>i G B n>i :\<n<k,l<i<C\ (A<)i<<<c) = J] A *( B «,*) 

l<n<fc,l<i<C 

is a function of (Aj)i<i<c conditionally to which A and (A"„ ! i)„>i i \<i<c are thus 
independent. Since A G £ is arbitrary, we deduce that £ C cr((X„ ! i) Tl >i ! i<i<c) and 
(^n,i)n>i, i<i<c are conditionally independent given (Ai)i<i<<7, which implies £ C 
cr((Ai)i<i<c) a.s. This proves the first statement, from which the second follows since 
T is a.s. trivial if the X n ^ are independent, see the Kolmogorov 0-1 Law. 

4. Concluding remarks 

The important bound (|10[) is a combinatorial estimate of the difference between 
sampling with and without replacement, see Aldous [2 Prop. 5.6 p. 39] and Diaconis 
and Freedman 4, Theorem 13 p. 749] for related results. It is used in 0] to prove the 
de Finetti Theorem. 

Theorem[3]allows proving (Pi, . . . , Pf)-chaoticity results by use of Proposition^and 
Sznitman's compactness-uniqueness methods for proof that the empirical measures A^ 
converge in law to Pi for 1 < i < C. This was the main motivation for this paper, as 
can be seen by its title. In the reviewing process, the referee's suggestions lead to a 
much improved and fuller study of multi-exchangeable systems. 

The techniques developed in this paper could also extend convergence results, such 
as Kallenberg [5J Theorem 1.3 p. 25] and Aldous [TJ Prop. 7.20 (a) p. 55], suited for a 
family of multi-exchangeable systems of fixed possibly infinite class sizes depending on 
a parameter. We refrain do to so for the sake of coherence. 

Acknowledgments 

We thank the anonymous referee for a careful reading and many stimulating remarks. 
He helped us rediscover the depth and beauty of Aldous's treatise pQ. 

References 

[1] Aldous, D.J. (1985). Exchangeability and related topics. In Ecole d'ete de Probabilites de Saint- 
Flour XIII - 1983, Lecture Notes in Mathematics 1117, Springer, Berlin, pp. 1-198. 

[2] Bellomo, N. and Stocker, S. (2000). Development of Boltzmann models in mathematical 
biology. In Modelling in Applied Sciences: A Kinetic Theory Approach, eds N. Bellomo and 
M. Pulvirenti, Birkhauser, Boston, pp. 225—262. 

[3] Cercignani, C, Illner, and R., Pulvirenti, M. (1994). The mathematical theory of dilute 
gases. Applied Mathematical Sciences 106, Springer, New- York. 

[4] Diaconis, P. and Freedman, D. (1980). Finite exchangeable sequences. Ann. Prob. 8, 745-764. 

[5] Graham, C. (1992). McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with 
discrete jump sets. Stoch. Proc. Appl. 40, 69-82. 

[6] Graham, C. (2000). Kinetic limits for large communication networks. In Modelling in Applied 
Sciences: A Kinetic Theory Approach, eds N. Bellomo and M. Pulvirenti, Birkhauser, Boston, 
pp. 317-370. 



Chaoticity for multi-class systems and echangeability within classes 



<) 



[7] Graham, C. and Robert, Ph. (2008). Interacting multi-class transmissions in large stochastic 
networks. Preprint. larXiv:0810.0"347l hal:inria-00326156 

[8] Grunfbld, CP. (2000). Nonlinar kinetic models with chemical reactions. In Modelling in 
Applied Sciences: A Kinetic Theory Approach, eds N. Bellomo and M. Pulvirenti, Birkhauscr, 
Boston, pp. 173-224. 

[9] Kallenberg, O. (1973). Canonical representations and convergence criteria for processes with 
interchangeable increments. Z. Wahrscheinlichkeitstheor. Verw. Geb. 27, 23—36. 

[10] Kingman, J.F.C. (1978). Uses of exchangeability. Ann. Prob. 6, 183-197. 

[11] Meleard, S. (1996). Asymptotic behaviour of some interacting particle systems; McKean-Vlasov 
and Boltzmann models. In: CIME summer school Montecatini Terme 1995, eds D. Talay and 
L. Tubaro, Lecture Notes in Mathematics 1627, Springer, Berlin, pp. 42—95. 

[12] StruCKMAIER, J (2000). Numerical simulation of the Boltzmann equation by particle methods. In 
Modelling in Applied Sciences: A Kinetic Theory Approach, eds N. Bellomo and M. Pulvirenti, 
Birkhauser, Boston, pp. 371—419. 



[13] Sznitman, A.S. (1991). Topics in propagation of chaos. In Ecole d'ete de Probabilites de Saint- 
Flour XIX - 1989, Lecture Notes in Mathematics 1464, Springer, Berlin, pp. 165-251. 



